|
The bag-of-words model is a simplifying representation used in natural language processing and information retrieval (IR). In this model, a text (such as a sentence or a document) is represented as the bag (multiset) of its words, disregarding grammar and even word order but keeping multiplicity. Recently, the bag-of-words model has also been used for computer vision. The bag-of-words model is commonly used in methods of document classification, where the (frequency of) occurrence of each word is used as a feature for training a classifier. An early reference to "bag of words" in a linguistic context can be found in Zellig Harris's 1954 article on ''Distributional Structure''. == Example implementation == The following models a text document using bag-of-words. Here are two simple text documents: (1) John likes to watch movies. Mary likes movies too. (2) John also likes to watch football games. Based on these two text documents, a list is constructed as: ( "John", "likes", "to", "watch", "movies", "also", "football", "games", "Mary", "too" ) which has 10 distinct words. And using the indexes of the list, each document is represented by a 10-entry vector: (1) (2, 1, 1, 2, 0, 0, 0, 1, 1 ) (2) (1, 1, 1, 0, 1, 1, 1, 0, 0 ) where each entry of the vectors refers to count of the corresponding entry in the list (this is also the histogram representation). For example, in the first vector (which represents document 1), the first two entries are "1,2". The first entry corresponds to the word "John" which is the first word in the list, and its value is "1" because "John" appears in the first document 1 time. Similarly, the second entry corresponds to the word "likes" which is the second word in the list, and its value is "2" because "likes" appears in the first document 2 times. This vector representation does not preserve the order of the words in the original sentences. This kind of representation has several successful applications, for example email filtering.〔 抄文引用元・出典: フリー百科事典『 ウィキペディア（Wikipedia）』 ■ウィキペディアで「Bag-of-words model」の詳細全文を読む スポンサード リンク
|